Add appearance-descriptor sidecar (sim half of multi-operator re-id) - #101
Merged
Conversation
A seeded, pixel-free, GT-derived appearance descriptor for cross-camera re-id: EntityAppearance (unit-norm descriptor keyed by entity_id) and AppearanceTable with a classmethod sample(entity_ids, dim, separation, seed). Descriptors are drawn deterministically (numpy default_rng, fixed draw order), unit-normalized anchors blended toward their shared mean by (1 - separation) and renormalized, so the separation knob monotonically controls inter-entity cosine similarity (1.0 near-orthogonal/separable, 0.0 collapsed/indistinguishable). The descriptor does NOT encode the GT entity id; it models OSNet-style confusability, and separation is the dose-response axis for re-id eval. Exported from the package top level.
Scene.appearance: AppearanceTable | None = None. Same additive contract as possession/activity: None by default and never read by the manifest builder, so the byte-golden analytic manifest is unchanged whether or not the sidecar is present.
Covers determinism across 3 seeds (identical descriptors for identical inputs, distinct-but-valid unit vectors across seeds), monotonicity of the separation knob (mean pairwise inter-entity cosine similarity strictly increases as separation falls 1.0 -> 0.0), unit-norm and dim-shape invariants, empty and single-entity edge cases, and that the descriptor does not encode the GT id. Byte-golden: a scene with the sidecar attached produces a manifest byte-identical to the same scene without it, and the manifest never contains appearance GT.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A seeded, pixel-free, GT-derived appearance-descriptor sidecar so a cross-camera fuser can disambiguate subjects that geometry alone cannot (the collinear / ambiguous case). This is the sim half of multi-operator re-id; the rt
SimTableDescriptorconsumer follows.What it is
src/multicam_sim/appearance.py, mirroring the additive-sidecar pattern ofactivity.py:EntityAppearance(BaseModel, frozen):entity_id: str,descriptor: list[float](unit-norm).AppearanceTable(BaseModel, frozen):dim(default 512),separation(0..1),seed,entries, plusAppearanceTable.sample(entity_ids, dim=512, separation=..., seed=...). It draws one per-entity anchor withnumpy.random.default_rng(seed)in a fixed order, unit-normalizes, blends each anchor toward the shared mean by(1 - separation), and renormalizes.Attached to
Sceneasappearance: AppearanceTable | None = None.Additive / opt-in, byte-golden safe
Noneby default and never read by the manifest builder, so the analytic manifest is byte-identical whether or not the sidecar is attached. Two tests prove it: the manifest bytes with and without the sidecar are exactly equal, and the manifest never contains appearance GT.The separation knob = the re-id dose-response axis
separationmonotonically controls the mean pairwise inter-entity cosine similarity:1.0gives near-orthogonal, perfectly separable descriptors (the easy case),0.0collapses every descriptor onto a shared mean (indistinguishable). Sweeping it traces how re-id accuracy degrades as identities become harder to separate. Monotonicity is verified across 3 seeds.Pixel-free, GT-derived, does NOT leak the GT id
No pixels are rendered or read; the descriptor is a synthetic stand-in for a learned re-id embedding and models OSNet-style confusability. It does not encode and cannot be inverted to the ground-truth
entity_id(a test renames entities and confirms the vectors are unchanged), so the fuser is not handed the answer.Checks
uv run pytest -q,ruff check,ruff format --check, andmypy srcall pass.